Delete the trigger comments nothing needs any more#56
Conversation
A PR driven through a dozen rounds collects a dozen "@coderabbitai review" comments and a dozen acknowledgements, which buries the conversation a human came to read. crq now removes its own half of that as rounds progress, and `crq tidy` does it on demand. Deleting a comment crq still reads is how this becomes expensive, so a comment has to clear three guards. It belongs to a round that has PROGRESSED — a live round keeps its command, because that is the comment crq adopts instead of posting another. The bot acted after it, so it was read rather than merely old. And it predates the current head, because adoption only ever considers commands newer than the head commit; delete one of those and the next pump posts a duplicate and buys a second review. Only comments crq posted. Candidates are the command IDs recorded on its own rounds, never anything matching the text — a person's "@coderabbitai review" is their decision to ask, not crq's to erase, and crq posts under the same account so authorship alone cannot tell them apart. Never the bots' own comments. An auto-generated reply can be a rate-limit or skipped-review notice, which crq classifies as evidence and surfaces as a finding, so deleting those would quietly destroy feedback nobody had read yet. A recorded command that is already gone counts as tidied rather than failing: the bot removes some of its own command comments, and a person may have tidied by hand.
|
Warning Review limit reached
Next review available in: 27 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (22)
📝 WalkthroughWalkthroughAdds stale review-trigger comment detection and deletion, with dry-run reporting, automatic autoreview cleanup, a new ChangesTidy cleanup flow
Sequence Diagram(s)sequenceDiagram
participant CLI
participant ServiceTidy
participant Engine
participant GitHub
CLI->>ServiceTidy: Tidy(repo, pr, dryRun)
ServiceTidy->>GitHub: load rounds, comments, and PR activity
ServiceTidy->>Engine: evaluate stale commands
Engine-->>ServiceTidy: return spent comment IDs
alt dry run
ServiceTidy-->>CLI: return JSON deletion report
else apply
ServiceTidy->>GitHub: delete spent comments
ServiceTidy-->>CLI: return JSON result
end
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
To use Codex here, create an environment for this repo. |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@codex review |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1c57adf00
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/engine/tidy_test.go`:
- Around line 11-63: Replace the separate tests in
TestStaleCommandsKeepsWhatCrqStillReads,
TestStaleCommandsNeverTouchesALiveRound, and
TestStaleCommandsWithoutAHeadStillRequiresEvidence with one table-driven test
covering live, unanswered, head-cutoff/adoption, and stale-command outcomes.
Give each case its own TidyInput and expected stale IDs, then run them through a
subtest loop while preserving the existing behavior expectations.
In `@internal/engine/tidy.go`:
- Around line 53-54: Retain candidates when the current-head timestamp is
unavailable: update the stale-command filtering around the HeadAt guard in
internal/engine/tidy.go lines 53-54 so a zero HeadAt produces no stale commands
rather than skipping the candidate. Update the corresponding expectation in
internal/engine/tidy_test.go lines 52-62 to verify the command is retained.
In `@llms.txt`:
- Around line 212-213: Update the user-facing command-removal contracts in
llms.txt (lines 212-213) and skills/coderabbit-queue/SKILL.md (lines 176-177) to
state that the command must predate the current head when that timestamp is
known, preserving the existing recorded, progressed, answered, and non-bot
conditions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: df17a142-421e-4d07-b2c0-5a7d4cc03d5e
📒 Files selected for processing (9)
cmd/crq/main.gointernal/crq/auto.gointernal/crq/config.gointernal/crq/tidy.gointernal/crq/tidy_test.gointernal/engine/tidy.gointernal/engine/tidy_test.gollms.txtskills/coderabbit-queue/SKILL.md
📜 Review details
🧰 Additional context used
📓 Path-based instructions (3)
internal/engine/**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
internal/engine/**/*.go: Keepinternal/enginepure: decision logic must receivenow, perform no I/O, context, or GitHub access, and table-test every rule.
DecideFiremust be the single fire owner and evaluate guards in order: open, readable/current head, eligible phase, primary review unavailable, already reviewed, free slot, account quota, minimum interval, then adopt/post.
Keep convergence, fire, and co-reviewer rules ininternal/engine; keep orchestration ininternal/crq, and table-test engine rules.
Files:
internal/engine/tidy_test.gointernal/engine/tidy.go
internal/crq/**/*.go
📄 CodeRabbit inference engine (AGENTS.md)
Keep
internal/crqlimited to orchestration and effects wiring;service.gois the only effects executor for CAS writes andPostIssueComment, andDryRunmust write nothing.
Files:
internal/crq/config.gointernal/crq/auto.gointernal/crq/tidy_test.gointernal/crq/tidy.go
internal/crq/config.go
📄 CodeRabbit inference engine (AGENTS.md)
Add a default entry in
parseCoBotswhen a new co-reviewer should be enabled by default.
Files:
internal/crq/config.go
🔇 Additional comments (3)
cmd/crq/main.go (3)
227-248: LGTM!
381-381: LGTM!
535-559: LGTM!
A rate-limited round retries, and every retry posts a NEW "@coderabbitai review" — the bot answered the previous one with the rate-limit notice, so it can never be adopted again. crq overwrote CommandID and forgot the old comment, which is why a throttled PR collects a column of identical requests: nothing knew they were there, and tidying could only ever find commands from rounds that had finished. A round now records the commands it supersedes, bounded so a PR that retries all day cannot grow its round without limit, and tidy treats them as spent whichever phase the round is in. They are exempt from the predates-the-head check on purpose: crq's own record that it has posted a successor is stronger evidence than any timestamp. Observed on PR #50 — five heads, nine review requests, three of them for the first head alone. Codex is not affected: it gets one command per head because crq skips the trigger when the bot has already reviewed it.
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
|
@codex review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75bffe7186
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
Adoption was the hole. A round records the command it adopted in exactly the same CommandID as one it posted, so "the command IDs crq recorded on its own rounds" included a person's "@coderabbitai review" — and once the head moved on and the bot had spoken, tidy would delete it. With a maintainer token nothing even objects. A round now records the comments it WROTE, with the reviewer each was addressed to and when it landed, and that list is the only candidate source. Adopting a comment is not writing one. It also replaces the spent-command list: a command missing from the round's current anchors is one the round replaced, which is the same fact without a second field. Four things fall out of carrying the reviewer with the comment: * a co-only round's anchor is the co-reviewer's trigger, not the primary's — unrelated CodeRabbit activity no longer passes for the answer that trigger is still waiting on, and the id stops appearing on the deletion list twice; * a superseded co-reviewer trigger is attributed to the bot that was asked rather than to CodeRabbit; * candidates are filtered against the comments actually on the PR, so a comment tidy already deleted is not DELETEd again on every later pass with the 404 read back as a fresh removal; * state is re-read after the observation and before the deletes, so a round another fleet member started in that window keeps its command. An unreadable head commit now keeps every non-superseded command instead of clearing the guard: the command may well be adoptable again once the read recovers, and deleting it buys a second review. And a round that completes while holding the slot ("cleared") or inside the reviewing sweep is tidied — between them that is most successful rounds, so the automatic cleanup was barely running.
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@codex review |
|
@coderabbitai review |
✅ Action performedReview finished.
|
This branch was cut before #51, so it still read the reviewer configuration off the Service: observe with no config argument, and coCommandBodies as a Service method. Both are Config methods now, and the branch stopped compiling the moment main moved under it — the tests here passed only because CI builds the branch, not the merge. Tidy takes a Config through the whole path now: which comments count as a trigger depends on who reviews, so that has to be a value the caller supplies. It reads the fleet configuration today, and per-repo reviewers (#57) can substitute one there in a single line without threading anything new through.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc15556832
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b14786357d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fbac5bf6fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 71bb2ce844
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@codex review |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7ba62de67
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
A PR driven through a dozen rounds collects a dozen
@coderabbitai reviewcomments and a dozenacknowledgements, which buries the conversation a human came to read. crq now removes its own half of
that as rounds progress, and
crq tidy <repo> <pr> [--dry-run]does it on demand.What it will not touch
Deleting a comment crq still reads is how this becomes expensive, so the rules are the feature.
Only comments crq posted. Candidates are the command IDs recorded on crq's own rounds — never
anything matching the text. A person's
@coderabbitai reviewis their decision to ask and not crq'sto erase, and since crq posts under the same account, authorship alone cannot tell the two apart.
Only from rounds that have progressed (your rule). A live round keeps its command, because that
is the comment
DecideFireadopts instead of posting another one.Only after the bot answered it, so it was read rather than merely old.
Only if it predates the current head. Adoption only ever considers commands newer than the head
commit; delete one of those and the next pump sees no command, posts another, and buys a second
review of the same code.
Never the bots' own comments. An auto-generated reply can be a rate-limit notice or a
skipped-review notice, which crq classifies as evidence and surfaces as a finding. Deleting those
would quietly destroy feedback nobody had read yet — which is the one outcome worse than a noisy PR.
Where it runs
Automatically as rounds progress under
crq autoreview(CRQ_TIDY=0disables), hooked to the pumpresult so it costs one observation on a PR crq was already looking at rather than a sweep of the
fleet. Housekeeping failures are logged and never break the pass that did the real work.
A limitation worth stating
Only commands crq still holds a round record for are candidates, and
Archiveis bounded — sotrigger comments from long-trimmed rounds stay. Widening it would mean matching on text, which is
exactly the step that risks deleting a human's comment, so it stays narrow.
A recorded command that is already gone counts as tidied rather than failing: the bot removes some of
its own command comments, and a person may have tidied by hand.
Verification
Exercised live on this repo.
--dry-runfirst, then real: #48 lost six spent triggers while the liveround's own
@coderabbitai review(18:42) and@codex reviewsurvived, along with every bot commentand every human one. Across #50–#55 it removed 16 and correctly reported "no progressed round has a
trigger comment to remove" for the PR that had none.
TestStaleCommandsKeepsWhatCrqStillReadscovers each guard in the pure rule.TestTidyRemovesOnlySpentCommandsCrqPostedstages a spent command, the live round's command, aperson's request and a bot reply, and asserts only the first goes.
TestTidyDryRunWritesNothingpins that a dry run reports the same decision and changes nothing.gofmt -l .clean,go vet ./...clean,go test ./... -count=1green.Ref #42
Summary by CodeRabbit
New Features
crq tidy <repo> <pr>to remove spent review-trigger comments posted bycrq.--dry-runto preview comments that would be removed without making changes.CRQ_TIDY=0.crq.Documentation